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l|5 -£ - To iaprbve tbe accuracy of the inforaal reading 

tli:n¥e:nt6ry IIBI) ,.a differential set of crit«ria is nicessary for both 
ilo^^^ and coapfehe^sion sccxea f or differen€ levels and 

IrlSdiLngr^vCdirid^^^^^ recognition scored . 

5>?sKouW''Sfefle<:t-'\OT^^^ .errorsr of i.-nse£€ibns^ '-oiissibns, - ^ 
^^I^MoaMciations, sabstiti^^ unkoiin vords^ and transpositions; 

:i^<:^fpt$iatic^beli4j!rior shoald. not be considered. .After the student has 
l^^eidl'^coiprehe^isi^ be asked on a iitexal level, on 

^"%i|lid^1irmnd^standih on vocabulary, and cn evaluative ski:^l; all 
KllMji Se liioliy. cb^^ criteria for deteraining 

, oSsat'isfiicAofy ;.r in coaprehension, word 

>^>xtcogl^^OBi> and \8yaptoiiBic^b«iavior ; cioaprehension is the aost 
,^figilfi:ciit in^^eterainin^^^^^^^^ ia«k of the IBI is 

S^^;pilkce K he s t i ideu t at his leadlu y le » el thla decijiiocnU-Tirst— 
f^Allition v^ guiBihtitative data, then an error analysis giv^s 

llfSlalttt^v^ inforaatibn. iliceaent precedes'' analysis , but both are 
l^te^cel^f X?f or effective lreading« diagnosis and piaceaent. (Df ) 
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MEASURING .READING PERFORMANCE INFORMALLY 

William R. Powell 
University of Florida 



In the very early days of motion pictures when Western thrillers were 
shown in the ranch cowns of the Wes^, real cowboys would ride in from miles 
around to see the Saturday'night show. As they sat in dingy theaters, 
' they would watch the villain ride across the screen shooting uprthe town, 
robbing banks, and holding lovely heroines at his mercy. It is said that' 

9 

the cowboys sometimes became so excited ^nd enraged at the villain that 
they- would pull out their own guns and, shoot at the screen. 

^he absurdity of this effort is readily apparent. If they had wanted 
to get the villain they would have done better to turn in their^seats and 
^stioot o ut tbe^-pyojec^-n, — Tbe-soi^eeft-wa^-ojrty- theHf^^eetton- of the p r oblem. 



/ 




The problem^as somewhere else, difficult to get to, but no -less reaV. 

Reading professionals may be spending their time shooting at screens. 
At a reading conference in Ch^icago, Johns (1976) asked 24 professionals 
their views regarding 24 given statements about the informal reading 
inventory. It is interesting that those- reading people were most concerned 
that the IRI overestimated, the proper reading level for instruction (18 
of 24 respondents). Howeyer, statements showing concern about the validity, 
neliability; the criteria for determining levels v/ere ranked 15th, 16th, 
and ]7th respectively. There was apparent^ little awareness on the part. of 



these; professionals that the problem for which they were most concerned was 



a mere reflection of the things they rated so relatively Insignificant- 
The informal reading inventory (iRI) is a teacher constructed, test 
which can Ue used to measure oral .reading performance. It is a record of 
, a, reader's performance in a sudcession of short '(125-175 words) reading 
passages of increasing difficulty. Steuver (1967) has shown that it takes 
a. passage length in this range for the quantitative errors to stablize 
*s6 a subsequent qualitative analysis may be done, if desired. 

the record is obtained by having a student -read each passage while 
^ the teacher monitors the* oral reading performance ancf asks previously 
prepared, varied questions about each passage. .The teacher keeps a record 
of the oral reading 'errors and responses to the questions. The reading 

passages -Should be taken from or should be equivalent in difficulty to 

• . - / f * 

books used for instruction in the classroom. 

-K The, purpose of an !RI is to allow a teacher to make a semi-cpntrolled 



observation of the many characteristics of a child's readiiig performance. 

The fesiiits of an IRI should predict^ the book level, readability level, or 

■ • , • " ■ ■ ' • • - ^ , \ » 

-level of difficulty a. subject can process' at tr.iree levelsxpf performance 

according Ato . spec fft^^ It is certain character isttcs of observed 

reafdingrbehav.ior and. th^ criteria which are the. focus of this 




of bipal; •'^ea^^^^^ reveal. that 

pattern 
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I^^'* • .tor enror than In increasingly more difficult material . There is an 

||^;<v ..observed pattern of error ratios from high error ratios to low ones, from 

\ jprodutjbi number of roiscues.^ (Powell, 1970; 



PpweH-^d: punkem^ 1971; Dunkeld, 1970;, Page, 1975; Spache, 1963; Durrell, 
^"^^'■Z':' , J;?,5.5;;-GrlTO^^ Millar, 1978.) 



1^.^^^^ : t]);i s he^^^^ error pattern-produces a differential effect of observed 



^ , ;actura;cjf^ t?V?ls> Performarice in first-grade material 

11^:1 : ^iC ndt^the^^^-'S^^^^^ in, sixth, grade material. Error values of 

,^s|feii^^^ not the same for each level * There is an^ 

/"^.av^enagfa of ^ twa-androne-haTf percent difference between ^levels as 

tlieAma^^ in.. difficulty, this: pattern , creates an inverse 

~ /^^^^^^^ o'f diff1cui|y and latitude for cerrpr in ^ 

Slf?^? /^ " 7; ' acceptable .ranges, of ,perf6rmanca. (Powell , 1970;. Dunkfeld, 1970) , 

IIMS^ ' t.^v :TH^ key-^^^^^^^^^^ across different reading levels' is the dif- 



ISj^KV / /ficuUy or .the readability of the material/ Difficulty or maturity of 



, written language, not the grade .placement or age^of the reader, is the 

:|r4m^r^4'actor ,e error :production. (Powell, 1971; Watkins, 1953.) 

S|^.K1n^^^^ has observed that when a ygungster experiences difficulty 



.in/i^adi^ of , his errors represents- the level of diffii:jjlty of 

MM^>;. tliie/^niiterial (expressed fn^recidabjlity grade level estimates) at. which he 



MC - than his, chrprtplogical grade .placement y., chronological 

i5^ ---^^ji-CA" - - ^ - " - - - - .... . . 



^r^^v^^: ;;;./ the^fiTdt^ie^ the material .the more error that can be tolerated and still 
l^l^ij.^^^ As: the material increases In difficulty, 

if^^C/ '-J'^^^ understanding. Students in material 

^^ii^Si-s^tf" v-^- Vv--' : f:,/, .-^ ■?-\\ ' , - ~; . , ^ 
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unflemanding than in, material from^the three to five range, which -in turn 



?^l.-v pre-^Hm/r to skond grade difficulty can cope with more error and. maintain 

l%v- - '/ /' ■ .'\ • ■ 

. ^ unflemanding than in 

can, tol'erate more eKror than in material Of sixth grade difficulty or 
^^'•i>' -.- ifcigher. Readers tolerate error in an inverse relationship to the diffi 



.//c/l^^^^^ the matenal. (Powell., 1971; Page, 1975; Dunkeld,^ 1970.) 
^|v>v / Support for these findings also comes from growth in cognitive devel- 



-vppment- -studies,,. 'Wohlvtili (.1?62.) indicates that. the movement from perception 
. ;fd/.;reaspning. perfnits,.^^^^^^ of behavior on information 

,- frpmXhe~in^ stimulus field. This movement' i§ due to a decrease in 
.r^edundancy, an. increase in selectivity,, and -^n increase in the temporal 
-and- spatiaircontiguity tolerated, (pp;. 98-102) 

.. Wheh>a student reaches, apprpxlrnately the fifth grade instructionaT 
ptVC.-, ' reading level, ~ii 4s doubtful if the criterion- of word recognition accuracy 
,. . is:- a. valid i hd^iefeorHt^i^ i li e di.rr.bVeMlidtiun Thrrs-iTnrfl7ignvX§~pparent 
. .4r{.§unk%rd.k.;da$a, 0 and Malejcek7(l;976) i'ouhcL^a, plateau effect at 
li^w?-- - • ' -g^-pi te^^ the phonetically consistent Czech language. -Once 

III^Js&>- -^^^ to. a level of automaticity,, then that 

W^^^^r'.H ' factor- ibWins.;^^^ .nihction. As -a read6r, increases in 

f#S'SvV^ .reaWnd^^ qraduaJly loses its strength as an 



S^SK.t;.^'?^^^^^^ represent a. relationship to. 
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. (l?73h then it is possible to calculate the cumulative word corpus for 



feetc'h' grade level data available. If the calculation is made at the 

- • " *- ' , , 

,poAht,.wher,e frsutration. .level would begin, then the number of words that 
,;shdut{j form the, base, s.ight vocabulary for a given grade leveT can be 
Iftf J /.>j'.-./'fdet(Bi&mlnud-. - For example,, at third grade leyeT, a decrease in the error 
. ra't|a ah- increase in sight ybcabulary by :abput 275 words. 

r-.Therlfore^pne^ does :make .a difference/a each additional error 
%Uld .r|f/!.^^^^ sight, ybcabulary of approxitfiately 250 to 300, 



J'i-V 



|%fl t ',: ' the .typ.es qflmiscues 'that are i:Aunte,d. as errors in computing the 



llf^r . ' iword.-riacognition ratio makes a difference, too. the evidence, from Dunkeld's 
.-/.stuyy ,(1974i would indicate thai oriiy , insertions, omissions, misprdnun- 

. J./ -S IV'' ' ■ - 

r • .s.piat^oris.j..S*.Ubst1=tu^ unknown words, and transppsitions should be counted 
l^v? : - ;as^ ,sc6r^ bl e^ .errors . The inclusidn of repetitions^ seff-cprrectipns, 

.', - .hesitations, qr any other type of syrnptpmalic, behavidr does not enhance 
^'"li^f.-^ ...-1-1-; 'j|h§.4ccU.ca,cy of word recognition scpref. HdWever, aSf.Cjay.<1972) has so 



fM*V , cleanly indicated,- the number of self-rcorreQtions to, the totaVffamber of 



Hf/i^fV, . errors are of diagnostic value in revealing the efficiency of processing 
S^v^ v/.,! cl^jes.. This diagnostic i-nsight is.additlphal to the word recpgnitior\ 



Itlfe: j^J^^^^^^^^ yfa an Informal reading inventory, 

V^^^l,;;!;!^^ the comprebensipn score on the ^ 

S^?*^^ Wi^^^ lilfplv tovbe'an unreliable score, as observation 




CPoweTl ' . *^ 



^ . . . hesitations, finger point, wbrd-by-word reading, holding book too close, 
^etc. ). are of diagnostic value only. during the^ first two years of the 



^ • • : . learning to read prbcess. (Duhkeld, 1970.)^ After that point, the 



_ - tjemporaiiily induced behavior fades into habituatiori and loses its diag- 



.5 \ ^ '.hostic significance; 



. Comprehension scores ^.on^an informal reading inventory has a differ-. 
. ential criteria in proportion to the reducing effect of word recognition. 
.Studiies h shown a w positive; relationship between word recognition 
arid cbmprehensiort. Ehri (1978) finds children must *'read words not only 
accurately but also rapidly and with 'attention;" (p. 18) Pace and 
^,Gol1nkoff (1976) state;|that although some reading theorists argue that ^ 
accurat^B v/ord recognition 's, net even necessary for the comprehension of 
the text, they conclude from their data, that if a word cannot be decoded 
Mf;;^ „ xea^jlyo'ts J^P^^^^ be retrieved and access of meaning will not 

occur,, (p. 765). ^Comprehension may be impaired If too many words Ccinnot ^ 
be^^d'e^^^ (p> 766). keying cpmprehehsiDn. deficiency is at 



' . ^ be,^diecoded- r 



|ea§t pa^ntia^^^^^ a -fai:lure to fully^ identify a sufficient 



piw&!^ ^ ' -'/^W^^^^ co^^se; of reading;. - 

ir.V ,|.HeAdi\tar§i!yeri :m^studies' by; ^larkinari; (1977) ar 



and ;Pace (1978)' clearly 
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a differential word recognition score should produce differential values 
for comprehensnon..perforniance. 

Repeated performance on the same material produces less error on 

• successive readings. (Powell, 1973; Busbopm, 1973; Page, 1975; Kasdon; 
T970; Lowell, 1970. There is a reduction factor which the second and 
:third reading of the material produces on the error effect. • The second 

. reading reduces en^_injvgrd_it^^ ^5 
tb\33-pjercent; a third reading and'^other related practice further reduces 
•error another 15 to 17 percent. Coupled with the error reducing effect of 
introducing vocabulary effectively prior to the first readina, it is 
'\ apparent that effective teaching can transform a given, piece of material 
from near frustra,tion^ level to .ihclependent level. Therefore, a different 
set of evaluative criteria is necessary for diagnosis (oral at sight) , a 
developmental lesson (silent before oral ) , and the completion process of 
, teaching a give'n selection. '* 



. The Criterion issue ^ - 

The above characteristics of reading behavior should be reflected in 
.any given set of criteria for evaluating reading performance. It is 
obvious that a uniform criterion cannot be applicable to the conditions 
of the word recognition and comprehension process. A differentiiil set of 
values, ?i,s jndicated for both, word recognition and comprehension scores. 
. . , ,v Jhe Betts, .(l94d) cri.teri^ for the IRI is by far the most widely used 
.. . MTiJes^^^^^ behavior. While the Betts criteria is 

l?|f^v ' J. , . .:ap|:llcab'l:e for a .given grade, rangje v fourth) and under developmental con- 



Powell 
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ditions at that Level, it obviously^ cannot evaluate reading performance 
. for, the la^ge number, of ceaders outside that range or under different ^ 

» - 

conditions. A differential set of criteria is essential for different 
levels and different conditions of reading. One se^ no longer will apply. 
Yet^when one reads* the descriptive accounts on the IRI in recent scholarly 
journals br^newly released tiextbooks in reading, a traditional one non- 



difi^rehtia.l set 6'f criteria continues 4:o be promulgated. As the frontier 
: editor says in the film The'Mah Who Shot Liberty Valance , "When there Is a 
■'•c6nflict"b"etweenrthe^^ the legend." 

To effectively evaluate reading behavior informally, three pieces of 

information ;must b^ known and used for a reliable and -valid measure: 0) 
„w!iatlto_locludeJ.n_^f fee ting the^crite (2) what not to include in the 
.^primary or. firstrorder evaluation; jand* (3) what baseline criteria from 

;vvhi,ch t6 form one's decision aboiit'the quality of reading. . 



. - . ;Ih. evaluating, quality of oral reading, the evidence (Dunkeld, 1970) 
suggests, that .orily errors ^f Insertions,., omissions., mispronunciations, 
substiitutioiikj unknown words, 'and trarispbsitions be included in determining 
\th,e^.wofd; rjecpg^^^^^ ratio, Errors of repetition, punctuation . 

' ' .omissjons,,,setf?correc,tions, proper ,na^^^ and places., dialect, arid speech 
idef.ects; :|hpUl;d -be; treated ^as s^iriptpritatic type errors and not used to 
.r . eff|cjt:4 the)', may be recorded for secpndary 

, . , •ariill'lsj^^^^^ "O-t i ncl u^ed; In: d^^^^^^^ the 

f^y^'-.^'i'^'^ finJtia^V^iuaLl^^^ ^Np assumption sh^^^^ be made that although 

SS&i/- V. 1 ^^^^ egMal .ipprtan but as yet no 
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satisfactory method has been .discovered to adequately weigh them properly. 

Symptomatic type errors should not be used -to effect the determination 

of word recognition accuracy. Fluency errors and observable behavior errors 

while distracting and probably desirable behskviors to apply extinction- to, 

a re/not primary determiners in assistin'g iK predicting the correct level 

of ^placement into instructional material. 

. the third essential item of information for measuring reading informally 
> 

is to,,have some framework or guidel-ines by which to decide, when a reading 
perfdrmahce. is moving toward the .unsatisfactory end pj th_e reading con- 
tinuum. Criteria can be applied along three dimensions: comprehension, - 
word r^cogri.ition, and symptomatic behaviors. These dimensions are important, 
in that order, for determining the final decision as to the quality of 
reading^- -A-di*scussit)n-oi^eiic|pcrit^?ioir^i^ deserves Attention. 

The gbmprehension dimension is the most important of the three in 
"deteFfinFi ng final ' pTci cemen t . When comprehension drops below the criterion 
for a given level, then it* matters not what happens on the other two 
dimensions. The reading should end and a finol judgement can be rendered. 
Th'^e comprehension questions asked following a reading (oral or silent) 
^should be varied according to type. There should be questions on a literal, 
.level, implied types of questions, questions on the vocabulary used in 
that passage, and questions requiring evaluative skill. In all instances, 
the questions should.be context dependent, i.e., answerable only by the 
'irtf'ormatipn base suppljjed in the passage and dependent upon it. If the 
questions ca^^ by prior experience or 1<nbwledge, then 

treadjng/cqniprehen i a that selection Is not to have been measured. 
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The criteriV-for comprehension by clustering reading l.evels'is grven 

» ' * - 

in Table 1, There is a differential effect by level and by conditio^?* In 

> materials at the r^^adabiTity -levels of grades one and two,^ a 55 percent 

\ ' , . , ' : ' \ 

\ ' ^" - ^ o ' V — 

" V - Rl ace" Table^l About Here 



criterion is adequate for Instructional placement wheji reading oral at 
sight i(diagnostib function) • For'difficulty levels three, *four, and five, 

60 percent is, the minimum satisfactory .level, and for readability levels 

* ' ^ It. 

six or higher, a 65 percent criterion- is essential to maintain quality of 
perfor:mance, ■ ' ■ * • 

These data are derived by interacting cloze criterion scores .from the 
Bormbuth data' (1972), the post oral cloze scores of Page (1975), the 
maxiinuiii cloze reconstruction figures (Weaver, 1977), with the word recog- 
^n\ti(rn figures; g^i yen Tffl^o^ scores 

> i ^ ' ' ^ . ' 

areL quf\e differ e^^ from the traditional 75 percent criterion (which was 
derived frqm;,early school bqards^ :crjterion for examining and licensing 
;teachers5.:i;e.,,. ar^^^^ more realistic than the 

jarbitrary.per^^^ Without :any : particular; precedent. A cloze score 

^blse-'has i?^ ac^t'ual .p.enforiiiance. Extrapolating ^ 

thefft^lfito,;^^ diif feren^ral, :per^^^^^ framework .pfoy ides a more realistic 
\eya'l|at^^^^ - — ^ ^ 

. ^ J iie:;Wq;r.^^^e^^ in tattle, T has previously been 

ipubll<jhed? and.- discussed (Powell?,. 1970.; 197.X;; 1973; and :1976) . While the 
|-^j)?#>^ .is ,:giVe"n'iri:'reddabi.ll^^^ clusters (:r^2v 3'-5;;,;6+:)- for ease of . 



• r 



TABLE 1 



n^er 



Informal , Reading Inventory 
Scoring Criteria By Performance Level 
and' Condition . 





- Diagnosis ^ 


'Developmental - 
Teaching^ 


tresson ' 
Evaluation; 




\VI/R- 


Comp. 


W/R-' 


Comp.- 


W/R 




Increp;ehder>t ^ 
''Level 














1 - 2- ' 


: 1/17+ 


80+^ 


. V17+ 


80+ . ■ 


.1/17+ . 


[ 88+=^ 


3-5 ' 
■ » ■ ♦ 


1/27+ . ,* 


85+ 


1/27+ ' 


85+ 


1/27+ 


-.85+ 


• or 


,.1/35+ , . 


90+- - 


. 1/35+; 




1/35+ 




Instructional 
' Level ■ 


« • - ' 


^ 






t * 


» 

t 


1 t 2- 


l/8c 1/16- 
1/13 1/26* 


-55-80 
60-85 , 


1/12 1/16 
1/20 - 1/26 


70-80 
75-85 


'Gmwerted to 
, Indepetident 




1/18 - 1/35 


65-90 


1/2C - 1/36 


80-l>J 

> * 


oLevel 

.f 


J 

* 


:Frustr&t:ion 
- 'level 7 ^ 




♦ 




t 

\ f 






' \^ 1 - 2' . • ' 


1/7 - 


. 55-. - 




: 70- 








^1/12- ■ 


6Qr-'' . 


'.1^19- 


.75-..* 


\ 






iy'17 -. ^ 




■ 1/25*-- 






r 



I 

% > 
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ij)pj4cat^^^ in practice, i^t , should be remembered that each level of diffi- 
<ul:tyjhai_i;ts, ,owh d^^^^ criterion figure .(^ee Powell , 1970)* The 

;readabiji^ ane fof* convenience only of closely related figures. 



■ . ,^fee is a. distinct S?reak jn the .patterVrof scores between readability 



it 



tevels: »twb .and three; and again between levels five and six. These natural 
weak^ reading growth form fhe cluster patterns, given in Table 1. 



'Mjr^x,^'^^^ ' \ Wfien .symptoms of difflcuTty are of value (in grades one and two), one 
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pr two sjnTiptonis^ark^.pTO the instructional . zone for 

fe^^^r: " " . ?;thbse>children. More symptoms present, than that are likely to represent 
the frustration level Qf reading while no observable symptoms would repre- 
sent the viridepen^^ ^ v 

The initial task of measuring^ reading informally is to^place the 
reader .into material , of the correct difficulty or readability level. This 



decision: .is f^irst based: on quantitative data.- Then an error analysis 
fl^;^ . • (Mcteoi; 1918;, Monroe, 932; Goodman and Burke 1972) is perf;ormea to, make 
ll^fe:; , a;. 'qu^^ Rerfetti, .Bell',, and Goldman (1976) make the general 

f • " xonelusion "that the differences connectecl with reading skills are quanti- ~ 
- .'ta# /e>{rathe»? .t^^^ (p. 12) .f hp issue is not one of either- 



, ■" br-,4but avmatter.;bf the pu information 
t^^^};J;v/C^-^^^^^ which;the data on a given reading 

|5 5g j J' ; ^^^^ fpfacement ,.p^ analys^i s,;, quantitative 

^ft^r."77Sr'^^e&.q^^^^ ^bd!th :are necessary for an effective diagnosis of 
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